The technology described herein relates to the processing of computer graphics, and in particular to a method of and an apparatus for performing, inter alia, multisampled graphics processing.
As is known in the art, graphics processing is normally carried out by first dividing the output to be generated, such as a frame to be displayed, into a number of similar basic components (so-called “primitives”) to allow the graphics processing operations to be more easily carried out. These “primitives” are usually in the form of simple polygons, such as triangles.
The graphics primitives are usually generated by the applications program interface for the graphics processing system, using the graphics drawing instructions (requests) received from the application (e.g. game) that requires the graphics output.
Each primitive is at this stage usually defined by and represented as a set of vertices. Each vertex for a primitive has associated with it a set of data (such as position, colour, texture and other attributes data) representing the vertex. This data is then used, e.g., when rasterising and rendering the vertex (the primitive(s) to which the vertex relates) in order to generate the desired output of the graphics processing system.
Once primitives and their vertices have been generated and defined, they can be processed by the graphics processing system, in order, e.g., to display the frame.
This process basically involves determining which sampling points of an array of sampling points covering the output area to be processed are covered by a primitive, and then determining the appearance each sampling point should have (e.g. in terms of its colour, etc.) to represent the primitive at that sampling point. These processes are commonly referred to as rasterising and rendering, respectively.
The rasterising process determines the sample positions that should be used for a primitive (i.e. the (x, y) positions of the sampling points to be used to represent the primitive in the output, e.g. scene to be displayed). This is typically done using the positions of the vertices of a primitive.
The rendering process then derives the data, such as red, green and blue (RGB) colour values and an “Alpha” (transparency) value, necessary to represent the primitive at the sampling points (i.e. “shades” each sampling point). This can involve, as is known in the art, applying textures, blending sampling point data values, etc.
(In graphics literature, the term “rasterisation” is sometimes used to mean both primitive conversion to sample positions and rendering. However, herein “rasterisation” will be used to refer to converting primitive data to sampling point addresses only.)
These processes are typically carried out by testing sets of one, or of more than one, sampling point, and then generating for each set of sampling points found to include a sampling point that is inside (covered by) the primitive in question (being tested), a discrete graphical entity usually referred to as a “fragment” on which the graphics processing operations (such as rendering) are carried out. Covered sampling points are thus, in effect, processed as fragments that will be used to render the primitive at the sampling points in question. The “fragments” are the graphical entities that pass through the rendering process (the rendering pipeline). Each fragment that is generated and processed may, e.g., represent a single sampling point or a set of plural sampling points, depending upon how the graphics processing system is configured.
A “fragment” is therefore effectively (has associated with it) a set of primitive data as interpolated to a given output space sampling point or points of a primitive. It may also include per-primitive and other state data that is required to shade the primitive at the sampling point (fragment position) in question. Each graphics fragment may typically be the same size and location as a “pixel” of the output (e.g. output frame) (since as the pixels are the singularities in the final output, e.g. display, there may be a one-to-one mapping between the “fragments” the graphics processor operates on (renders) and the output pixels, e.g. of a display). However, it can be the case that there is not a one-to-one correspondence between a fragment and an output (e.g. display) pixel, for example where particular forms of post-processing, such as downsampling, are carried out on the rendered image prior to displaying the final image.
(It is also the case that as multiple fragments, e.g. from different overlapping primitives, at a given location may affect each other (e.g. due to transparency and/or blending), the final pixel output may depend upon plural or all fragments at that pixel location.)
Correspondingly, there may be a one-to-one correspondence between the sampling points and the pixels of a display, but more typically there may not be a one-to-one correspondence between sampling points and output (e.g. display) pixels, as downsampling may be carried out on the rendered sample values to generate the output pixel values, e.g. for displaying the final image. Similarly, where multiple sampling point values, e.g. from different overlapping primitives, at a given location affect each other (e.g. due to transparency and/or blending), the final pixel output will also depend upon plural overlapping sample values at that pixel location.
One problem that is encountered when processing graphics for display (when displaying computer generated images) is that the displayed image is quantised into the discrete pixel locations of the display, e.g. screen or printer, being used. This limits the resolution of the image that is displayed and can produce unwanted visual artifacts, for example, where the resolution of the output display device is not high enough to display smooth lines. These effects are commonly referred to as “aliasing”.
FIG. 1 illustrates such aliasing effects. The left-hand side of FIG. 1 shows the image to be drawn, and the right-hand side shows the actual image that is displayed. As can be seen, the desired smooth curve of the white object in fact has a jagged appearance on the display. This is aliasing. (In FIG. 1, each square represents a pixel of the display, and the crosses represent the points at each (x, y) pixel location for which the colour value for that pixel location is determined (sampled). For example, the pixel A in FIG. 1 is drawn as all white, because the colour sampling point for that pixel location falls within the white object. It should be noted that in FIG. 1 only sample crosses on the pixels of interest are shown, although in practice all pixels will be sampled.)
All aliasing artifacts that could be visible to the naked eye could be removed by using a sufficiently high resolution display. However, the resolution of electronic screen and printers is typically limited, and so many graphics processing systems use other techniques to try to remove or reduce the effects of aliasing. Such techniques are typically referred to as anti-aliasing techniques.
One known anti-aliasing technique is referred to as supersampling or oversampling.
In such an arrangement, there are plural sampling points (positions) per pixel of the final display, and a separate colour sample is taken for each (covered) individual sampling point (e.g. by rendering each sampling point as a separate fragment). The effect of this is that a different colour sample is taken for each sampling point of a display pixel that is covered by a primitive during the rendering process.
This means that plural colour samples are taken for each pixel of the display (one for each sampling point for the pixel, since a separate colour value is rendered for each sampling point). These plural colour samples are then combined into a single colour for the pixel when the pixel is displayed. This has the effect of smoothing or averaging the colour values from the original image at the pixel location in question.
FIG. 2 illustrates the supersampling process. In the example shown in FIG. 2, four sampling points are determined for each pixel in the display and separate colour samples are taken for each sampling point during the rendering process. (Each such sample can accordingly effectively be viewed as a “sub-pixel”, with each pixel in the display being made up of four such sub-pixels.) The four colour value samples (sub-pixels) for a given pixel are then combined (downfiltered) such that the final colour that is used for the pixel in the display is an appropriate average (blend) of the colours of the four colour samples taken for the pixel.
This has the effect of smoothing the image that is displayed, and, e.g., reduces the prominence of aliasing artifacts by surrounding them with intermediate shades of colour. This can be seen in FIG. 2, where the pixel A now has two “white” samples and two “black” samples and so is set to 50% “white” in the displayed image. In this way, the pixels around the edges of the white object are blurred to produce a smoother edge, based on, e.g., how many samples are found to fall on each side of the edge.
Supersampling in effect processes the screen image at a much higher resolution than will actually be used for the display, and then scales and filters (downsamples) the processed image to the final resolution before it is displayed. This has the effect of providing an improved image with reduced aliasing artifacts, but requires greater processing power and/or time, since the graphics processing system must in effect process as many fragments as there are samples (such that, e.g., for 4× supersampling (i.e. where 4 samples are taken for each display pixel), the processing requirements will be four times greater than if there was no supersampling).
Other anti-aliasing techniques have therefore been proposed that, while still providing some improvement in image quality, have less processing requirements than full supersampling. One common such technique is referred to as “multisampling”.
In the case of multisampling, multiple sampling points are again tested for each display pixel to determine whether a given primitive covers the sampling points or not when the image is rasterised into fragments (at the rasterisation stage). Thus the sampling point coverage of a primitive in a multisampling system is determined in a similar fashion to a “supersampling” system (and so the positions of the outer geometric edges of the primitives are still effectively “supersampled” (oversampled) in a multisampling system).
However, in the rendering process of a multisampling system, all the sampling points for a given display pixel that are covered by the primitive in question are allocated the same single, common set of colour value data (rather than each having their own separate set of data as would be the case for supersampling).
Thus, in multisampling, plural samples are again taken for each pixel that will make up the final display, but rather than determining a separate colour value for each sample when rendering the “pixel” for a given primitive (as would be the case for a full supersampling system), a single colour value is determined and applied to all the samples for a display pixel that are found to belong to the same primitive in the final image. In other words multisampling calculates a single colour value for a given display pixel for a given primitive in the scene, which colour value is applied to (reused for) all samples (subpixels) of the display pixel that are covered by that primitive (in contrast to supersampling where a separate colour value is determined for each sample).
Because only a single colour value is provided for multiple samples for a given display pixel, multisampling is less processing intensive than supersampling and therefore can allow faster processing and performance than supersampling. However, there is a reduction in the quality of the displayed image as compared to supersampling, since although objects' (primitives') edges are still sampled at a higher resolution, colours are not.
Furthermore, although only a single colour value is provided for a multisampled fragment, graphics processing operations (e.g. rendering operations) are still performed in respect of each sample of the multisampled fragment (i.e. on a “per sample” basis). For example, when blending a multisampled fragment which represents four samples, four blending operations are carried out, one in respect of each of the samples. This is because, for example, the colour value that is to be blended at each sample position may differ, e.g. because the sample positions have different colour values associated with them in the tile buffer (e.g. in a colour buffer) as a result of rendering other (earlier) primitives.
Notwithstanding this, many graphics processing systems use a multisampling anti-aliasing technique, as multisampling can in general provide adequate (and improved as compared to when there is no multisampling or supersampling at all) anti-aliasing in a rendered image, but without all the extra processing and computational burden that full supersampling entails.
The Applicants, however, believe that there remains scope for improved techniques which provide for further reductions in the amount of processing required to process sampling points in, inter alia, multisampled graphics processing pipelines.
Like reference numerals are used for like components where appropriate in the drawings.